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REMARKS 

Applicants have studied the Office Action dated June 15, 2005. No new matter has 
been added, tt is submitted that the application, is in condition for allowance. 
Applicants have amended independent claims 1,11, and 14 and claims 3, 4, 8, 11, 13, 
14, 16 and 21. By virtue of this amendment, Claims 1, 3-11, 13-14, and 16-22 are 
pending. Reconsideration and further examination of the pending claims in view of the 
above amendments and the following remarks is respectfully requested. In the Office 
Action, the Examiner: 

• Rejected claims 1, 3-10, 14, and 16-22 under 35 U.S.C. 103(a) as being 
unpatentable over Takiguchi et al. (U.S. Patent No. 6,549,681), and further in 
view of Parulski et al. (U.S. Patent No. 6,812,961); and 

• Rejected claims 11 and 13 under 35 U.S.C. 103(a) as being unpatentable over 
Takiguchi et al. (U.S. Patent No. 6,549,681), and further in view of Parulski et al. 
(U.S. Patent No. 6,812,961), in further view of Examiner's Official Notice. 

Overview of the Present Invention 
The present invention provides a picture stitching device, a method, and a computer 
readable medium for creating a panoramic image from at least two images. A digital 
camera is one type of picture stitching device that has become very popular for creating 
panoramic images. To provide panoramic photography effects, digital cameras can 
interface with personal computers for joining together two or more images into one 
image to provide a panoramic effect by joining edge boundaries of images. One 
problem often encountered when using digital cameras for creating panoramic pictures 
is the amount of additional memory needed to save the panoramic image separately 
from the original images composing the panorama image. 

Another problem encountered is how the panoramic image is stored. Specifically, the 
99-RB-183 7 10/053,222 



PACE 0/18 • RCVD AT 0/15/2005 5:02:44 PM [Eastern Daylight Time] • 8VR:USPTO-EFXRF-6/36 • DNIS: 2738300 ■ C8tt):581 980 0812 " DURATION (mm-ss): 08-20 



09715/2005 16:57 561-989-9812 



FLEIT KAIN ET AL. 



PAGE 10/18 



prior art systems store the panoramic image in strips or in a format other than the 
complete panoramic image. A panoramic image format that is stored in strips or in a 
format other than the complete image is useful only for a stitching application. Other 
applications, such as, an electronic photo album, process each strip as a single image 
and displays each image separately. This displaying of each strip separately many 
times confuses the user since the entire panoramic image is not displayed at once. 
Accordingly, a need exists for a method and a system to store still images in an image 
format that is compatible with other image formats such as JPEG. 

An additional problem is that pictures saved with original parameters such as motion 
estimation, color correction, focal length and other image stitching parameters are not 
compatible with popular storage formats such as JPEG (Joint Photographic Experts 
Group ISO standard 10918). In fact, along with the Graphic Interchange Format (GIF) 
file, the JPEG is a file type supported by the World Wide Web protocol. Accordingly, a 
need exists for a method and apparatus to store images with panoramic parameters in 
such a way that the image storage format is compatible with other still image formats 
such as JPEG. In addition to the above problems, a user is left with very little choice for 
correcting imperfections in the panoramic image after it is captured if the parameters of 
motion estimation and color correction are lost 

To overcome the problems of using a digital camera to create a panoramic image, the 
present invention receives a first image forming a part of a series of images to form a 
panoramic image. The first image is then stored in memory. One or more subsequent 
images are received and for each of the images received one or more panoramic 
parameters are calculated between a current image and a previous image stored in 
memory. Also, the current image is stored with the one or more panoramic parameters 
in memory. The one or more panoramic parameters are also stored in a field of a 
compressed image format reserved for at least one of comments and extensions so as 
to preserve the stored image file compatibility with other images stored without the one 
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or more panoramic parameters. 

In order to more particularly point the distinctness of the present invention the following 
language has been added to the independent claims, i.e., claims 1,11, and 14 as 
follows: 

receiving a first image forming a part of a series of images to form a 
panoramic image; 

storing the first image in memory; 

receiving one or more subsequent images and for each of the 
images received performing the sub-steps of 

calculating one or more panoramic parameters including motion 
estimation and color correction between a current image and a previous 
image stored in memory; and 

storing the current image with the one or more panoramic 
parameters in memory, wherein the one or more panoramic parameters 
are also stored in a field of a compressed image format reserved for at 
least one of comments and extensions so as to preserve the stored image 
file compatibility with other images stored without the one or more 
panoramic parameters. 

No new matter has been added. Support for this amended language is found in the 
application as originally filed and particularly at FIGs. 10 and 13-14 and pages 13, 16, 
and 20. 



Rejection under 35 U.S.C. S103fa) as being unpatentable over Takiquchi and Parulski 
As noted above, the Examiner rejected claims 1, 3-10, 14, and 16-22 under 35 U.S.C. 
103(a) as being unpatentable over Takiguchi et al. (U.S. Patent No. 6,549,681), and 

99-RB-183 9 10/053,222 



PACE 11/18 ' RCVD AT 9/13/2005 5:02:44 PM (Eastern Daylight TlmeJ • 8VR:USPTOEFXRF4tt6 • ON18:2738300 " C8ID:561 989 9812 ■ DURATION <mm-ss):0*-20 



09/15/2005 16:57 561-989-9812 



FLEIT KAIN ET AL. 



PAGE 12/18 



further in view of Parulski et al. (U.S. Patent No. 6,812,961). Independent claims 1,11 
and 14 have been amended to distinguish over Takiguchi taken alone and/or in view of 
Parulski. The Examiner on Page 3 of the Office Action correctly states that Takiguchi 
teaches a method for storing a plurality of still images in which data is stored in JPEG 
data format. Takiguchi is silent on teaching storing panoramic data image including 
motion estimation and color correction for later stitching (or synthesizing) into a 
panoramic image. Takiguchi does not keep the motion estimation and color correction 
stored with images. Takiguchi may estimate these parameters after capturing using the 
"image synthesization unit 517 M but they are maintained and certainly not stored with 
the image. See Takiguchi at col. 19, line 15 to col. 22, lines 53. Accordingly, the 
present invention distinguishes over Takiguchi for at least this reason. 

The Examiner on Page 3 of the Office Action correctly states that Takiguchi is "silent as 
to storing additional data in a field resen/ed for at least one of comments and 
extensions" and goes on to combine Takiguchi with Parulski 1 The Examiner recites 35 
U.S.C. §103. The Statute expressly requires that obviousness or non-obviousness be 
determined for the claimed subject matter "as a whole," and the key to proper 
determination of the differences between the prior art and the present invention is giving 
full recognition to the invention "as a whole." The Examiner cites Parulski at col. 4, lines 
35-43, which is reproduced for convenience below with Emphasis Added. 

The processor 320 performs color interpolation followed by color and tone 
correction, in order to produce rendered sRGB image data. The rendered 
sRGB image data is then JPEG compressed and stored as a JPEG image 
file on the removable memory card 16. The JPEG file uses the so-called 
"Exif" image format defined in "Digital Still Camera Image File Format 
(ExIfT version 2.1. July 1998 by the Japan Electronics Industries 
Development Association (JEIDAV Tokyo. Japan. This format includes an 
Exif application segment that stores particular image metadata, for 



Applicants make no statement whether such combination is even proper 
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example the date and time the picture was captured, the lens f/number 
and other camera settings, and image captions or comments that can be 
selected or entered by the camera user 
Parulski is completely silent on panoramic stitching and synthesization. Parulski is 
directed to method and system for capturing images to a service provider. See Parulski 
Abstract and Summary. Parulski is silent on teaching storing panoramic data image 
including motion estimation and color correction for later stitching (or synthesizing) into 
a panoramic image. Accordingly, independent claims 1,11 and 14 distinguish over 
Takiguchi taken alone and/or in view of Parulski for at least reason. 

Moreover, the Federal Circuit has consistently held that when a §103 rejection is based 
upon a modification of a reference that destroys the intent, purpose or function of the 
invention disclosed in the reference, such a proposed modification is not proper and the 
prima facie case of obviousness can not be properly made. See In re Gordon, 733 F.2d 
900, 221 USPQ 1125 (Fed. Cir. 1984). Here the intent, purpose and function of 
Takiguchi is downloading images captured on a camera to a computer and running 
software application with an image synthesization unit 517. Takiguchi does not mention 
storing the panoramic image data including motion estimation and color correction with 
the images themselves. Therefore, the system in as described in Takiguchi taken alone 
and/or in view of Parulski must re-calculate the panoramic image information every time 
the images are to be synthesized. In contrast, the intent and purpose of the present 
invention is calculating and storing with each image the panoramic parameters including 
motion estimation and color correction with the image itself. Not only does the present 
invention eliminate the need to recalculate parameters required for panoramic image 
synthesization with the image, thereby reducing processing requirements, the present 
invention maintains well known file formats. By storing the motion estimation and color 
correction parameters in a field of a compressed image format reserved for at least one 
of comments and extensions so as to preserve the stored image file compatibility with 
other images stored without the one or more panoramic parameters . This maintains 
compatibility with both non-panoramic and panoramic images. This combination, as 
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suggested by the Examiner, destroys the intent and purpose of Takiguchi taken alone 
and/or in view of Parulski's use of calculating the motion estimation and color correction 
parameters every time the images are stitched or synthesized. Accordingly, the present 
invention is distinguishable over Accordingly, independent claims 1, 11 and 14 
distinguish over Takiguchi taken alone and/or in view of Parulski for this reason as well. 

Continuing further, when there is no suggestion or teaching in the prior art for storing 
motion estimation and color correction with the image itself the suggestion can not 
come from the Applicant's own specification. The Federal Circuit has repeatedly 
warned against using the Applicant's disclosure as a blueprint to reconstruct the 
claimed invention out of isolated teachings of the prior art. See MPEP §2143 and Grain 
Processing Corp. v. American Maize-Products, 840 F.2d 902, 907, 5 USPQ2d 1788 
1792 (Fed. Cir. 1988) and In re Fitch, 972 F.2d 160, 12 USPQ2d 1780, 1783-84 (Fed. 
Cir. 1992). The prior art reference Takiguchi taken alone and/or in view of Parulski 
does not even suggest, teach nor mention motion estimation and color correction stored 
with the image. The problems each of the cited references Takiguchi taken alone 
and/or in view of Parulski are trying to solve are different than the problem solved by the 
current invention. 

For the foregoing reasons, independent Claims 1, 11, and 14 as amended distinguish 
over Takiguchi taken alone and/or in view of Parulski. Claims 3-10, 13, and 16-21 
depend from amended Claims 1,11 and 14 respectively, and since dependent claims 
contain all the limitations of the independent claims, Claims 3-10, 13, and 16-21 
distinguish over Takiguchi taken alone and/or in view of Parulski, and the Examiner's 
rejection should be withdrawn, which withdrawal is respectfully requested. 

With regards to the group of claim 3, 13, 16 and 6 and 19, the Examiner again 
points to Parulski at col. 4, lines 35-43 for the teaching of Exif. Exif is defined as 
"(Exchangeable Image Format) Extensions to image file formats that hold 
the camera settings used to take the picture. Developed in 1995 by J El DA 
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for JPEG images, EXIF data was added to TIFF, RAW and other formats 
later. Most digital cameras support EXIF and save the data in the file 
headers. However, when an image is edited, the EXIF data may be 
automatically removed bv the software. An EXIF reader is a utility that is 
used to read, display and save EXIF data from a file ." 
From online URL (http://www.pcmag.com/encyclopedia_term with Emphasis Added). 
Accordingly, the Exif extension is an addition to the underlying compress image format 
standards JPEG and TIFF commonly used in cameras. This difference is clearly 
explained at online URL (http://en.wikipedia.org/wiki/Jpeg) with Emphasis Added: 
In computing, JPEG (pronounced jay-peg) is a commonly used standard 
method of lossy compression for photographic images. The file format 
which employs this compression is commonly also called JPEG; the most 
common file extensions for this format are .jpeg, jfif, .jpg, JPG, or JPE 
although Jpg is the most common on all platforms. 

The name stands for Joint Photographic Experts Group. JPEG itself 
specifies only how an image is transformed into a stream of bytes, but not 
how those bytes are encapsulated in any particular storage medium. A 
further standard, created by the Independent JPEG Group, called JFIF 
(JPEG File Interchange Format) specifies how to produce a file suitable 
for computer storage and transmission (such as over the Internet) from a 
JPEG stream. In common usage, when one speaks of a "JPEG file" one 
generally means a JFIF file, or sometimes an Exit JPEG file. There are, 
however, other JPEG-based file formats, such as JNG . 

JPEG/JFIF is the most common format used for storing and transmitting 
photographs on the World Wide Web. It is not as well suited for line 
drawings and other textual or iconic graphics because its compression 
method performs badly on these types of images (the PNG and GIF 
formats are in common use for that purpose; GIF, having only 8 bits per 
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pixel is not well suited for colour photographs, PNG can be used to 
losslessly store photographs but the filesize makes it largely unsuitable for 
putting photographs on the web). 

This distinction is important, because the present invention teaches compression format 
of JPEG/JFIF which is covered by the ISO DIS 10918 specification. See the present 
invention at page 15 of the specification as originally filed. Therefore a JPEG file format 
(also known as JPEG/JIFF file format) is interchangeable and is not the same as the 
Exit JPEG file format. 2 

Claims 3 t 13, 16 and 6 and 19 have been amended to clarify the JPEG file format "as 
defined bv the ISO DIS 10918" i.e. the JPEG/JIFF file format. Support for this 
amendment is found in the present invention as originally filed on pages 4 and 15. No 
new matter has been added. 

For the foregoing reasons, claims 3, 13, 16 and 6 and 19 as amended distinguish over 
Takiguchi taken alone and/or in view of Parulski, and the Examiner's rejection should be 
withdrawn, which withdrawal is respectfully requested. 

Rejection under 35 U.S.C. 5103(a) as being unpatentable 
over Takiguchi. Parulski & Official Notice 
As noted above, the Examiner rejected claims 11 and 13 under 35 U.S.C. 103(a) as 
being unpatentable over Takiguchi et al. (U.S. Patent No. 6,549,681), and further in 
view of Parulski et al. (U.S. Patent No. 6,812,961), in further view of Examiner's Official 
Notice.. Independent claims 1, 11 and 14 have been amended to distinguish over 

2 7JG based software writes a JFtFAPPO marker between SOI and Exif APP1 marker. 
According to the Exif specification, the ExifAPPI marker has to follow immediately after 
the SO/, just as the JFIF specification requires the same for the JFIF APPO marker! 
Therefore a JPEG file cannot legally be both Exif and JFIF at the same time !" 

99-RB-183 14 10/053,222 
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Takiguchi taken alone and/or in view of ParulskL The Examiner on Page 6 of the Office 
Action states that "Official Notice is taken that a program of instruction, executable by a 
machine and programmable directly into a machine, are easily transferred to a 
computer readable storage medium; a concept is well know that is executed in the art" 
However, for the reasons stated above in the section entitled "Rejection under 35 
U.S.C. §103(a) as being unpatentable over Takiguchi and Parulski " does ngi even 
suggest teach nor mention motion estimation and color correction stored with the 
image. The Official Notice taken by the Examiner does not suggest or teach this 
patentably distinct limitation either. Accordingly, independent Claim 11 as amended 
distinguish over Takiguchi taken alone and/or in view of Parulski and in view of Official 
Notice. Claim 13 depends from amended claim 1 1 , and since dependent claims contain 
all the limitations of the independent claims, claims 13 distinguishes over Takiguchi 
taken alone and/or in view of Parulski, and/or in view of Official Notice, and the 
Examiner's rejection should be withdrawn, which withdrawal is respectfully requested. 



CONCLUSIONS 

In light of the Office Action, Applicants believe these amendments serve a useful 
clarification purpose, and are desirable for clarification purposes, independent of 
patentability. Accordingly, Applicants respectfully submit that the claim amendments do 
not limit the range of any permissible equivalents. 

Applicants acknowledge the continuing duty of candor and good faith to the disclosure 
of information known to be material to the examination of this application. In accordance 
with 37 CFR §§ 1.56, all such information is dutifully made of record. The foreseeable 
equivalents of any territory surrendered by amendment is limited to the territory taught 
by the information of record. No other territory afforded by the doctrine of equivalents is 
knowingly surrendered and everything else is unforeseeable at the time of this 

(Emphasis added from online URL http://syIvana.net/jpegcrop/exifpatch.html) 
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amendment by the Applicants and their attorneys. 

Applicants respectfully submit that all of the grounds for rejection stated in the 
Examiner's Office Action have been overcome, and that all claims in the application are 
allowable. No new matter has been added. It is believed that the application is now in 
condition for allowance, which allowance is respectfully requested. 

PLEASE, if for any reason the Examiner finds the application other than in condition for 
allowance, the Examiner is invited to call either of the undersigned attorneys at (561) 
989-9811 should the Examiner believe a telephone interview would advance the 
prosecution of the application. 
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